Data mapping workflow

A data mapping workflow is a series of extraction instructions, called steps. These steps process and extract the data from the source and store them in records whose structure is determined in the Data Model (see The Data Model). Together with the data source settings, the Data Model, and the sample data, this is what makes a data mapping configuration (See Data mapping configurations).

The data mapping workflow is shown on the Steps pane at the left (see Steps pane).

Creating a data mapping workflow

A data mapping workflow always starts with the Preprocessor step and ends with the Postprocessor step. These steps allow the application to perform actions on the data file itself before it is handed over to the data mapping workflow (Preprocessor step) and after the Data Mapping workflow has completed (Postprocessor step).
When you create a new data mapping configuration, these steps are added automatically, but they don't actually do anything until you configure them.
In between the Preprocessor and Postprocessor step, the workflow can contain as many steps as needed to extract the required data.

Adding steps

Extracting data is the main way to build a data mapping workflow; see Extracting data.
Extract steps, Condition steps and Repeat steps can be added after selecting data in the Data Viewer.
All steps can be added via the Steps pane:

In the extraction workflow on the Steps pane, select the step after which to add the new step.
Right-click on the Steps pane and select Add a Step; then select one of the step types.

To add a first Step inside a Repeat or Condition structure, click on the initial step of the Repeat/Condition structure. This highlights the entire structure and any Step you add now would be positioned immediately after the structure. Click once more on the structure's initial step to highlight it alone. The next Step added will now be positioned as the first child inside the structure.

Editing steps

The properties of each step in the extraction workflow become visible in the Step properties pane when you select that step in the Steps pane.

The name of each step is shown in the Steps pane. You can change it under Description in the Step properties pane.

The other properties are different per step type; see Steps.

Rearranging steps

To rearrange steps, simply drag & drop them somewhere else on the dotted line in the Steps pane.
Alternatively you may right-click on a step and select Cut Step or use the Cut button in the Toolbar. If the step is Repeat or Condition, all steps inside it will also be placed on the clipboard. To place the step at its destination, right-click any step and select Paste Step, or use the Paste button in the toolbar. The pasted steps will be positioned below the selected step.

Keep in mind that steps may influence each other, so you may have to move other steps as well to ensure that the workflow continues to function properly. In a Text file for example, an Extract step may require a Goto step to be positioned immediately before it to move the current position to a certain location in the source data.

Deleting steps

To delete a step, right-click on it in the Steps pane and select Delete Step.

Testing the extraction workflow

The extraction workflow is always performed on the current record in the data source. When an error is encountered, the extraction workflow stops, and the field on which the error occurred and all subsequent fields will be greyed out. Click the Messages tab (next to the Step properties pane) to see any error messages.

To test the extraction workflow on all records, you can:

Click the Validate All Records toolbar button.
Select Data > Validate Records in the menu.

If any errors are encountered in one or more records, an error message will be displayed. Errors encountered while performing the extraction workflow on the current record will also be visible on the Messages tab.